The use of automated vision systems in traffic surveillance and monitoring applications has increased drastically over the last several years. Such applications include traffic flow monitoring, vehicle counting, and queue detection and intersection management. While there are cameras for surveillance or the like fixed on a stationary medium such as a fixed pole on the road for taking pictures or images of moving scenes, e.g., vehicles, pedestrians, and other motions, the cameras may be also mounted on a medium that could be moving, for example, a traffic light that is hung by a cable and that could swing, for instance, due to wind, other weather conditions, or any other condition that would cause the traffic light to move. If the cameras are installed on such moving mediums, the images taken from frame to frame need to be compensated for the motion of the camera. That is, the image recognition or computer vision techniques or the like need to be able to distinguish the moving objects in the scene from the object displacements occurring from frame to frame due to the moving camera. Therefore, it is desirable to have a technique for accounting for this motion.